Method and system for efficient fragment caching

ABSTRACT

Methods for serving data include maintaining an incomplete version of an object at a server and at least one fragment at the server. In response to a request for the object from a client, the incomplete version of the object, an identifier for a fragment comprising a portion of the object, and a position for the fragment within the object are sent to the client. After receiving the incomplete version of the object, the identifier, and the position, the client requests the fragment from the server using the identifier. The object is constructed by including the fragment in the incomplete version of the object in a location specified by the position.

RELATED APPLICATION INFORMATION

This application is a Divisional application of allowed U.S. patentapplication Ser. No. 10/622,209 filed on Jul. 18, 2003, now U.S. Pat.No. 7,114,032.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to distributed systems, and moreparticularly to a system and method for fragment caching in distributedsystems.

2. Description of the Related Art

Caching is an important technique for improving the performance ofdistributed systems, including Web-based systems. A significant problemwith caching is that not all data are cacheable. Data, which is dynamicand changes quickly, or personalized data, in which a Web page istailored to a specific user, may not be cacheable. Such a Web page maynot be usable by other clients (e.g., it may include the name of aspecific user). Therefore, caching a page may be of limited utilitysince other clients cannot use it, and other clients would need adifferent version of the page.

Other concerns include widespread changes that need to be made to updateWeb pages. Suppose, for example, that a particular fragment is containedin 2000 popular Web pages, which should be cached. Using theconventional approach, the cache would contain a separate version of thefragment for each page, resulting in as many as 2000 copies. Thisbecomes burdensome and expensive.

Therefore, a need exists for a system and method for more efficientlyupdating information in distributed information systems.

SUMMARY OF THE INVENTION

Methods for serving data, in accordance with the present invention,include maintaining an incomplete version of an object at a server andat least one fragment at the server. In response to a request for theobject from a client, the incomplete version of the object, anidentifier for a fragment comprising a portion of the object, and aposition for the fragment within the object are sent to the client.After receiving the incomplete version of the object, the identifier,and the position, the client requests the fragment from the server usingthe identifier. The object is constructed by including the fragment inthe incomplete version of the object in a location specified by theposition.

These and other objects, features and advantages of the presentinvention will become apparent from the following detailed descriptionof illustrative embodiments thereof, which is to be read in connectionwith the accompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

The invention will be described in detail in the following descriptionof preferred embodiments with reference to the following figureswherein:

FIG. 1 is a block diagram showing relationships between embeddedfragments and pages for illustrating the present invention;

FIG. 2 is a block diagram showing a client server system in accordancewith one embodiment of the present invention;

FIG. 3 is an illustrative list of identifiers and positions employed inaccordance with the present invention;

FIG. 4 is a diagram showing a Web page or other complex object, whichemploys a header and includes incomplete objects and fragments inaccordance with the present invention;

FIG. 5 depicts a message flow scenario in which header and incompleteobjects are cached, while fragments are retrieved from a server; and

FIG. 6 is a block/flow diagram showing methods/systems for efficientfragment caching in accordance with the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

One method, which avoids the need for updating entire data objects orlarge portions of data objects, is to compose complex data objects (e.g.Web pages) from simpler fragments. To efficiently serve the data, acache stores the fragments and composes them into complete entities. Afragment may recursively embed other fragments. The fragment approach isefficient because the overhead for assembling a Web page from simplerfragments is usually minor compared to the overhead for constructing thepage from scratch, which can be quite high.

In accordance with the present invention, fragment-based Web pagegeneration and caching has a number of advantages. To employ fragmentsto permit partial caching of personalized pages, the personalizedinformation on a Web page is encapsulated by one or more fragments thatare not cacheable, but other fragments in the page are.

When serving a request, a cache composes pages from its constituentfragments, many of which are locally available. Personalized fragmentshave to be created by the server. As personalized fragments may onlyconstitute a small fraction of the entire page, generating only thepersonalized fragments would require lower overhead than generating allof the fragments in the page.

The fragment-based approach also makes it easier to design Web sites.Common information that needs to be included on multiple Web pages canbe created as a fragment. To change the information on all pages, onlythe fragment needs to be changed. Generating Web pages from fragmentsprovides other benefits as well. When a particular fragment changes butthe rest of the Web page stays the same, only the fragment needs to beinvalidated or updated in the cache, not the entire page. Fragments canalso reduce the amount of cache space taken up by multiple pages withcommon content.

If the fragment-based method of page composition is used, only a singlecopy of the fragment needs to be maintained. Fragments can beconstructed to represent entities that have similar lifetimes. Fragmentcaching is advantageous when different parts of the same Web page havedifferent lifetimes. If such pages can be broken down into fragmentsbased on expected lifetimes, then when part of a page changes, it mayonly be necessary to fetch one or a few fragments instead of the wholepage.

Fragment caching also provides advantages for common informationincluded across several pages. Such information can be cached in asingle fragment as opposed to replicated in every cached page thatincludes the common information. For these situations, fragment cachingreduces the amount of cached information and the amount of informationsent from the server to the cache.

This invention disclosure describes a system and method for efficientlygenerating fragments at a server while encoding the proper informationfor a remote cache to efficiently compose a complete entity fromfragments. Some aspects of the present invention include the informationgenerated by the server, the communications protocol between the serverand cache, and fragment assembly at the cache.

The present invention will illustratively be described in the context ofconstructing HTML pages from fragments; however, the present inventionmay be applied to other types of data as well. The present invention isparticularly applicable to the Web; however other systems, which usecaches to improve performance, can benefit as well.

For Web-based caches, the HTTP protocol, as described in “HypertextTransfer Protocol—HTTP/I.I”, RFC 2626, R. Fielding et Al., June 1999,may be used. The HTTP protocol defines a number of header fields, whichare convenient for storing caching information used in accordance withthe present invention. HTTP headers will illustratively be used by thepresent invention. It should be recognized that other methods and otherprotocols other than HTTP may be employed within the spirit and scope ofthe invention.

It should be understood that the elements shown in the FIGS. may beimplemented in various forms of hardware, software or combinationsthereof. Preferably, these elements are implemented in software on oneor more appropriately programmed general-purpose digital computershaving a processor and memory and input/output interfaces. Referring nowto the drawings in which like numerals represent the same or similarelements and initially to FIG. 1, a set of Web pages, P1, P2, and P3,are depicted in which P1 and P2 include several fragments.

P1 includes a plurality of fragments f1-f5. Some fragments are embeddedin others and are depicted graphically as one fragment box inside ofanother, e.g., f2 is embedded in f5, f2 is also embedded in f1 which isembedded in f3. P2 includes fragments f2, f4 and f5 where f2 is embeddedin f5. These pages and fragments will be referred to throughout thisdisclosure to illustrate aspects of the present invention.

Referring now to FIG. 2, a client-server system 200, which employs thepresent invention, is illustratively shown. While the system 200 depictsa single server 202, cache 204, and client 206, the system 200 mayinclude a plurality of each of servers 202, caches 204, or clients 206.The arrows in the figure depict network communication paths between theentities. It is also possible to have direct network communication pathsbetween the server 202 and client 206.

Note that the server 202 may comprise additional entities other than aWeb server. For example, server 202 may also include additional back-endcomputers with databases, transaction processing capabilities, etc. ManyWeb sites include complex back-end processing systems, and server 202may include such systems.

Server 202 has the capability of generating Web pages from fragments. Asystem for generating complex Web pages from fragments is disclosed in“A Publishing System for Efficiently Generating Dynamic Web Content” byChallenger, lyengar, Witting, Ferstat, and Reed, Proceedings of INFOCOM2000.

Cache 204 may store either complete Web pages or fragments of Web pages.In some situations, server 202 may have the ability to explicitly storeinformation in cache 204. In other situations, client 206 requests anobject from cache 204. In the event that the object is not found incache 204, cache 204 requests the object from server 202.

Cache 204 may store some objects but not others. Server 202 may provideinformation indicating whether an object should be cached. If a cacheand server are communicating using HTTP, for example, then there aremethods within the HTTP protocol to specify whether or not an objectshould be cached.

Objects may have expiration times associated with them. An expirationtime indicates when a cached object is no longer current. After theexpiration time for an object has expired, cache 204 should not serve anobject without first checking with a server 202 that the object is stillvalid.

Some systems may also have the ability for server 202 to sendinformation to cache 204 that a cached object is no longer valid.Fragments may be cached remotely either by being pushed from a server oron a cache miss during a request for an object by a client.

When server 202 receives a request for an object, which includesfragments (e.g. a top-level HTML page), it may determine if the requestis coming from a cache 204, which recognizes fragments. If the serverand cache are using a protocol such as HTTP, this information could bestored in headers. Caches, which recognize fragments, may indicate thisin a request header. If cache 204 recognizes fragments, server 202 cansend an incomplete version of the object including the object minus itsembedded fragments. For top-level fragments included in the object,server 202 sends back an identifier 302 (FIG. 3) identifying thefragment (for Web fragments, this could be similar to a uniform resourcelocator (URL)) along with a position 304 (FIG. 3) indicating theposition of the fragment within the incomplete version of the object.Positions 304 may be specified in several ways, including but notlimited to offsets within the incomplete version of the object.Fragments may recursively embed other fragments. If a protocol such asHTTP is being used, identifiers 302 and positions 304 may be transmittedin header fields.

If server 202 determines that cache 204 does not recognize fragments,the server 202 can send the complete object (which may be, for example,a complete HTML page) to the cache. To reduce object assembly overhead,it is possible to pre-assemble complete objects at the server in advanceand maintain both the complete objects and incomplete versions withfragments at the server. This has the drawback of occupying more memoryspace. It may also introduce consistency problems for objects, which arefrequently updated.

When server 202 receives from a cache 204 a request for an object, whichis a fragment, included in another object, it is safe for the server toassume that the cache understands fragments. The server can respond tothe cache with the fragment along with identifiers 302 and positions 304for any fragments recursively embedded within the served fragment.

Suppose that a client 206 requests an object P1 composed of fragments.The request goes to cache 204, which recognizes fragments but whichcurrently does not include P1. The request is sent from the cache 204 tothe server 202. The cache 204 may indicate to the server (using arequest header, for example) that the cache recognizes fragments.Instead of passing P1 in its entirety to the cache 204 (which it wouldlikely do if the request were from a cache which did not recognizefragments), the server 202 sends P1′, a stripped down version of P1minus 1 or more fragments, to the cache 204. Identifier(s) 302 andposition(s) 304 are also sent to the cache corresponding to the one ormore fragments not in P1′. These fragments are preferably at the toplevel, e.g., they should be directly embedded in P1′. For fragmentswhich should be recursively embedded within a higher level fragment inP1′ but not directly embedded in P1′ it is not necessary to passidentifier/position information.

P1′ may be cached. To cache P1′, identifiers 302 and positions 304corresponding to fragments which should be embedded at top level in P1are stored in the cache, preferably but not necessarily ordered byposition. When cache 204 is called upon to serve P1, cache 204 obtainseach fragment f which needs to be embedded in P1′ to result in P1. Someof these fragments may already be cached. Other fragments (e.g., highlydynamic or personalized fragments) may have to be fetched from a remoteserver. Different fragments may have to be obtained from differentservers.

For each top-level fragment f embedded in P1, f is fetched either byaccessing f from cache or by fetching f from a remote server. If f isfetched from a remote server, specific information from a particularclient may be passed to the server 202 to permit, for example, creationof a personalized fragment. One method of sending such information forWeb-based system is via cookies.

Fragment f may recursively include other fragments. Persistentconnections may be used to fetch multiple fragments from the same serverin a single connection. For fragments on different servers without openconnections to a cache, the cache may either fetch the fragmentsserially or in parallel using multiple threads. If multithreading isused to fetch multiple fragments in parallel, buffering may be needed toreceive the fragments.

The cache 204 may begin responding to the client 206 immediately bysending parts of P1 which cache 204 has already stored. If fragments arebeing fetched serially, the cache may repeatedly serve data until thecache 204 reaches a position corresponding to the next unservedfragment. At this stage, the cache 204 checks whether the fragment iscached. If so, it obtains the fragment from cache. Otherwise, cache 204requests the fragment from a server.

Separating references to fragments from the body of P1 eliminates theneed for cache 204 to parse the body of P1 and improves the performanceof the cache. This is illustrated in FIG. 4.

Referring to FIG. 4, header information 402 describes the component andposition information to formulate a complete object 400 by utilizingincomplete object 404, inserting Fragment 406 at position f1, andinserting Fragment 408 at position f2.

FIG. 5 illustrates one possible flow of messages between client, cache,and server to achieve the delivery of the complete object 400,comprising incomplete object 404 and its associated fragments 406 and408, to the client.

Referring to FIG. 5, client 502 requests object 400 from cache 504. Inthis scenario, cache 504 has both header information 402 and incompleteobject 404 stored locally, while fragments 406 and 408 are not storedlocally. Cache 504 determines that fragments 406 and 408 need to berequested from server 506 and issues those requests. Cache 504 thendelivers bytes 0 through f1-1 (as specified in header 402) of object 404to client 502.

When fragment 406 is returned by server 506, the cache 504 deliversfragment 406 to client 502. Cache 504 then delivers bytes f1 throughf2-1 of incomplete object 404 to client 502. When these bytes have beensent, cache 504 delivers fragment 408 to client 502. When delivery offragment 408 is complete, cache 504 delivers the remaining bytes ofincomplete object 404 to client 502. The serving process for fragmentsis recursive, since a fragment may recursively embed other fragments.The cache may use a maximum time limit for fetching a fragment. Once thetime limit has expired, the cache will not try to waste additionalresources fetching other fragments in the object. It will insteadcomplete sending what information it can to the client quickly and/orrespond with an error message.

One problem, which can occur in fragment assembly, is when thespecification of fragment relationships erroneously specifies a cycle ininclusion relationships. For example, suppose that an entity erroneouslyspecifies that f1 includes f2, which includes f1. If a system attemptsto construct f1 without trying to detect cycles, it may enter into aninfinite loop. This may be prevented using multiple methods. If aparticular fragment f1 is included in an object, an embedding depth ofthe fragment may be defined as the number of inclusion relationshipsused to define the object's position in the object.

For example, consider FIG. 1, within objects P1 and P2, fragment f4 hasan embedding depth of 1, since f4 is included within P1 and P2 but nointermediate fragments also within P1 and P2 also include f4. Similarly,f5 has an embedding depth of 1. Fragment f1 has an embedding depth of 2within P1 because f1 is included in f3, which is itself included in P1.One occurrence of f2 within P1 has an embedding depth of 3, i.e., theone in which f2 is included in f1, which is included in f3, which isincluded in P1. A second occurrence of f2 has an embedding depth of 2,e.g., where f2 is included in f5, which is included in P1.

Under one method, the system uses a maximum embedding depth orthreshold, m. Different maximum embedding depths may be specified fordifferent fragments. If the system encounters a fragment with anembedding depth exceeding m, it can flag this as an error and return anappropriate error message and/or send a partially constructed version ofthe page without fragments embedded at levels beyond m.

Another method is to maintain a list of fragment identifierscorresponding to the inclusion relationships. A hash table can be usedfor storing the identifiers. Each time the list of inclusionrelationships increases by a fragment, the hash table is examined to seeif the fragment has already been encountered in the list. If so, it isconcluded that a cycle exists. If not, the fragment identifier is addedto the hash table, and the process continues. Once the recursive processof dealing with a particular fragment is complete, that fragment may beremoved from the hash table. This avoids the mistaken impression thatthere is a cycle when there is actually just multiple inclusion of afragment, as in, for example, FIG. 1, with f2 included twice in P1.Without removal of f2 when processing of its inclusion in f1 completed,f2 may still be in the hash table when its inclusion in f5 wasprocessed. That may lead to the mistaken impression that there was acycle, which is one reason that earlier removal is desirable. Other datastructures can be used (e.g. ordered lists, balanced trees) in place ofa hash table in this approach. Hash tables have the advantage that theycan be designed to be efficient. Note that this approach can be combinedwith maximum embedding depth thresholds. For example, if the length ofthe list exceeds a maximum embedding depth threshold, the system maystop trying to look for a cycle, even if none has yet been detected.

Graph traversal algorithms may also be applied to detect cycles ininclusion relationships. For example, fragments can be represented byvertices in a directed graph, and inclusion relationships can berepresented by directed edges in the graph. To determine if there is acycle in a list of inclusion relationships, a number of techniques canbe used which do not need to employ auxiliary data structures such ashash tables.

One such technique includes the following. The list of inclusionrelationships is traversed using two pointers. One pointer traverses onefragment at a time. The second pointer traverses two fragments at atime. If the two pointers eventually meet, then a cycle has beendetected. Graph traversal algorithms such as these do not have to beapplied after each new inclusion relationship is encountered. Instead,the system can set a threshold and not apply the cycle detection testuntil the number of inclusion relationships has reached the threshold.

If the threshold has been reached and the cycle detection algorithmfails to detect a cycle, the system can increase the threshold andcontinue looking for cycles. This method can set an upper limit on thesize of the threshold. In response to no cycle being detected after theupper limit is reached, the system can stop looking for a cycle.

Thresholds may be adaptively set to be just beyond the highest value ofthe embedding depth expected to be encountered.

Referring to FIG. 6, a method/system for serving data is depicted, whichincludes a method/system for determining if cycles exist in embeddedfragment sets and/or inclusion relationships are too long in length(depth) with regard to a threshold. The system/method includes in block602, maintaining an incomplete version of an object at a server andmaintaining at least one fragment at the server. The incomplete versionof the object and the fragment may be stored on a plurality of serversor on the same server. In addition, portions of the incomplete objectand the fragment may be stored on a plurality of servers. In block 604,in response to a request for the object, such as a Web page, from aclient (for example, a cache), the incomplete version of the object, anidentifier for a fragment including the object, and a position for thefragment within the object are sent to the client. In block 606, afterreceiving the incomplete version of the object, the identifier, and theposition, the fragment is requested by the client from the server usingthe identifier.

In block 608, the object is constructed by including the fragment in theincomplete version of the object in a location specified by theposition. The step of constructing includes determining whether a depthof inclusion relationships in the object exceeds a threshold in block610 and in response to the depth exceeding the threshold, abandoningconstructing the object in block 612. In block 614, the threshold isincreased and step 608 repeated until the object can be constructed,time expires or construction of the fragment is complete. Other actionsmay include, for example, simply sending an error message.

In block 615, a method/system for determining if cycles exist ininclusion relationships and/or if inclusion relationships of fragmentsare too long (exceeds a given depth) is shown. Several methods may beemployed for making these determinations. Two methods are indicated inblock 615 of FIG. 6 and are separated by a dashed line.

In block 616, a list of fragment identifiers corresponding to inclusionrelationships may be maintained. The list of fragment identifiers may beincluded in a hash table. The hash table may be examined each time theinclusion relationships change in block 618.

In block 619, the list of inclusion relationships may be examined whenthe list of inclusion relationships increases by a new fragment to seeif the new fragment has already been encountered in the list ofinclusion relationships. In block 620, if the new fragment has alreadybeen encountered, it is concluded that a cycle exists; otherwise, thenew fragment identifier is added to the hash table, in block 622. Inblock 623, the new fragment is removed when its time expires orconstruction of the fragment is complete.

In an alternate approach to blocks 616-623, in block 624, fragments maybe represented by vertices in a directed graph, and inclusionrelationships may be represented by directed edges in the graph. Inblock 626, it is determined if there is a cycle in a list of inclusionrelationships. This may include traversing a list of inclusionrelationships using two pointers, in block 628, wherein a first pointertraverses one fragment at a time, and a second pointer traverses twofragments at a time. If the two pointers meet in the graph, a cycle hasbeen detected.

Having described preferred embodiments of a method and system forefficient fragment caching (which are intended to be illustrative andnot limiting), it is noted that modifications and variations can be madeby persons skilled in the art in light of the above teachings. It istherefore to be understood that changes may be made in the particularembodiments of the invention disclosed which are within the scope andspirit of the invention as outlined by the appended claims. Having thusdescribed the invention with the details and particularity required bythe patent laws, what is claimed and desired protected by Letters Patentis set forth in the appended claims.

1. In a cache memory system comprised of a plurality of informationfragments in which a information fragment may include anotherinformation fragment, a method for determining whether a set ofinclusion relationships includes a cycle, comprising the steps of:examining the set of inclusion relationships to determine whether adepth of inclusions exceeds or equals a threshold; if the depth exceedsor equals the threshold, using graph traversal techniques to determineif a graph comprised of inclusion relationships includes a cycle byrepresenting fragments by vertices in a directed graph, and inclusionrelationships by directed edges in the graph; traversing a list ofinclusion relationship using two pointers wherein a first pointertraverses one fragment at a time, and a second pointer traverses twofragments at a time such that if the two pointers meet in the graph, acycle has been detected; if the depth is below the threshold, returningto the step of examining; and if using the graph traversal techniques,no cycle is determined, incrementing file threshold and returning to thestep of examining.